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California’s new accountability and continuous improvement 


framework relies on district and school leaders using multiple 
measures of school performance to identify where change is needed, 


California's Local Control Funding Formula (LCFF) requires districts to 
report multiple measures of student performance that reflect success in 
the goal of preparing students for college, career, and citizenship. As they 
engage in the Local Control Accountability Plan (LCAP) process, they 
are expected to use state and local indicator data from 
California's School Dashboard 1 to monitor student 
progress. When Dashboard indicators identify student 
subgroups as low performing or low growth, districts 
are encouraged to engage in a process of 
continuous improvement to develop strategies 
and then monitor their effectiveness." At this 
early stage of implementation, education 
leaders have an opportunity to learn from early 
adopters who are already using continuous 
improvement principles. In this case study, we 
share how Fresno Unified School District 
(FUSD) developed and utilized its data 
dashboard and the principles of 
Improvement Science to increase college 
access for their students, in partnership with 
the University of California, Merced. 


and to monitor carefully the development, testing, and evaluation of 


improvement strategies over time. This process of continuous improvement 


requires that local leaders have access to research-based evidence and strategies that 


they can implement in their schools and opportunities to learn from one another about what works, under which 


conditions, and for which students. PACE’s series of Continuous Improvement Briefs aims to support education 


leaders at all levels in learning how to improve the performance of their schools and students. 
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Fresno's Aim: Increase Student 
Awareness About Postsecondary 
Choices 

Since 2009 FUSD has invested in a robust data 
dashboard to support its school improvement work. 

The Fresno School Quality Improvement (SQM) and 
Targeted Action Index is made up of 75 indicators 
including standardized test performance, English 
Learner re-designation, measures of student growth 
mindset, measures of school climate, and college 
enrollment.'" 

Using this data dashboard, the district's Equity and 
Access team found evidence that many students were 
eligible to apply to a variety of California's colleges and 
universities, but most of them applied to just one. This 
pattern persisted even among the district's low-income 
students, for whom college application fees for up to 
four California State University (CSU) campuses and 
four University of California (UC) campuses were 
waived. 

Fresno’s Equity and Access team 
found that many students were 
eligible to apply to a variety of California’s 
colleges and universities, but most of 
them applied to just one. 

The FUSD team recognized this as an equity, access, 
and social justice issue that violated the district's 
guiding principle: "All students are given an equal 
opportunity to graduate with the greatest number of 
postsecondary choices from the widest array of 
options." 

Identifying and understanding this issue was the start. 

To resolve it, the team in Fresno searched for an 
approach to system improvement that could accelerate 
their efforts. They ultimately landed on Improvement 
Science. The Improvement Science framework, first 
used in the health care sector, is now being applied in 
education systems. It is defined by six principles :' 7 

1) Identify specific problems 

2) Focus on key participants 

3) Attend to variation in performance (what works, 
for whom, under what set of conditions) 

4) Reflect upon the existing system that is designed 
to produce current outcomes (systems thinking) 


5) Measure processes and outcomes to assess the 
efficacy of strategies 

6) Utilize rapid Plan-Do-Study-Act (PDSA) cycles to 
promote quick improvement 

The Improvement Science framework helps districts 
understand how to focus on specific problems, 
introduce small measureable changes, measure the 
impact of these changes, and determine whether and 
when these changes create true improvements that 
should be spread more widely across the system. 

Improvement Science helps 
districts focus on specific problems, 
introduce small measureable changes, 
measure the impact of these changes, and 
determine whether these changes create 
true improvements. 

FUSD is a leader in using the principles of Improvement 
Science to help solve problems of practice. Multiple 
teams within FUSD's central office are committed to 
applying some or all of these principles to their work. In 
this case study, we detail how the Equity and Access 
team has used the district's data system and a locally 
developed online tool to plan for and test 
improvements that were designed to increase the 
number of eligible high school students applying to 
multiple California colleges and universities. 

Identifying Specific Problems and 
Focusing on Key Participants 

Determining how to move forward in solving their 
problem of college access presented an opportunity 
for the Equity and Access team to think differently 
about how school districts normally approach the 
change process. Rather than jump directly into testing 
potential solutions, a team of data analysts turned first 
to existing district data to understand what was 
happening with their college-eligible students. 

Using eight years of historical data from FUSD's data 
dashboard and data retrieved from the National Student 
Clearinghouse, the Equity and Access team created a 
set of student academic profiles. Each profile included 
a list of colleges that were likely to accept a FUSD 
student with similar academic characteristics. Next, the 
FUSD team used their data dashboard to compare 
these profiles to current student data. This allowed 
them to identify eligible students who were not 
applying to CSU and UC universities where they likely 
would be accepted. ^ 




Attending to Variation in Performance and 
Reflecting on the Existing System 


and demonstrate that they were acquiring the skills and 
expertise necessary to translate Improvement Science 
into practice. 


Before developing and testing specific solutions to this 
problem, the Fresno team sought to understand the 
problem as it was experienced by the user, in this case 
the District's college-eligible students. The Equity and 
Access team worked closely with school counselors to 
understand the experiences of their students. They 
interviewed students to document (a) why certain 
students applied to more colleges than others and (b) 
what interventions had already been tried by 
counselors. This allowed district leaders to understand 
the variation in performance across schools, as well as 
challenges and opportunities for improvement. 
Afterwards, the team identified the following set of root 
causes that could help explain existing college 
application patterns in FUSD: 

• The district had never communicated to its 

students about which specific colleges and 
universities matched their academic profiles. 

• School staff and counselors did not have detailed 
knowledge about institutions of higher education 
outside of the Fresno area because most attended 
local colleges themselves. 

• School counselors had limited time to meet with 
students individually to discuss college plans. 

• Students and their parents had little knowledge 
about institutions of higher education outside of 
the area, and lacked crucial information about 
financial aid, application waivers, and other policies 
designed to help low-income students. 


The Cycle of Continuous Improvement Learning and 
Competency Tool, which is still in the testing stage, 
helped Fresno execute their Plan-Do-Study-Act model 
by creating a road map with detailed instructions for 
team members. The tool also served as a hub for 
documentation. 



Figure 1: Plan-Do-Study-Act Cycle of Improvement 


The Equity and Access team identified one root cause 
to focus on: the lack of students' awareness of their 
matched college options based on their academic 
profiles. 

Using PDSA Cycles and Measuring 
Processes and Outcomes to Assess 
Improvement 

Continuous improvement requires the measurement of 
processes and outcomes to determine whether tests of 
change are producing actual improvements to the 
performance problem. The goal was to increase the 
number of college-eligible students applying to CSUs 
and UCs that matched their academic profiles. The 
Equity and Access team relied on their own online tool, 
the Cycle of Continuous Improvement Learning and 
Competency Tool, to provide structure to the entire 
improvement process. This tool helped the team 
operationalize all six principles of Improvement Science 


Beginning with the Plan phase, the Fresno team 
developed an intervention by creating individualized "I 
Am Ready" packets for every senior who qualified to 
apply to CSU and UC campuses. This packet was 
intended to increase students' awareness of their 
individual college eligibility. These packets were mailed 
to current students who had been identified using 
course-taking, A-G, SAT/ACT, and other data as good 
matches to the alumni profiles of students who had 
already been accepted to a wide range of CSUs and 
UCs. 

During the Do phase, the team developed and 
assembled the college packets, which provided 
specialized information about the students' matched 
campuses based on their academic profiles. The 
packets also included application fee waiver 
information and a form to give to their school 
counselor to set up an appointment to discuss college 
applications. 




To support this effort, school counselors underwent 
two days of training to learn more about various CSU 
and UC campuses and student eligibility. The entire 
Do process was in contrast to typical counseling 
practice, in which counselors met with students to 
discuss course-taking and to provide college 
application resources in a much less comprehensive 
fashion and usually only in response to student 
request. 

Measurement and data collection were embedded 
into each step of the Do process. Counselors 
documented where students were applying as well as 
their reasons for not applying to eligible campuses. 
They documented the number of students who set up 
counseling appointments using the forms included in 
the packets. This data served as leading process 
indicators in the Cycle of Continuous Improvement 
Learning and Competency Tool (e.g., how many 
students used the form to schedule a counseling 
appointment?) and were used to inform future 
improvement efforts (e.g., what reasons did students 
give for not applying?). 

Sending “I Am Ready” packets and 
encouraging students to have 
conversations about them with school 
counselors helped to increase the number 
of students applying to UC/CSU outside 
of Fresno more than 50 percent. 

After the Do cycle, the FUSD team engaged in Study. 
They asked themselves whether their data showed an 
improvement in the performance problem. The team 
also studied their implementation of each component 
of their change idea and documented concrete 
examples of why specific components did not get 
implemented. Reasons included an inability to 
prioritize the work, a lack of clarity around the task or 
the knowledge and skills to complete it, and a lack of 
funds or decision-making authority to complete the 
task. These insights helped to improve the change 
idea the following year. 

After reviewing all of the evidence on their graduating 
students, the Equity and Access team determined that 
the packet idea was successful. Sending these packets 
and encouraging students to have follow-up 
conversations about them with school counselors 
helped to increase the number of students applying to 
CSU/UC outside of Fresno from 382 to 578, an 
increase of over 50 percent. Reviewing the 


implementation and effects of this change idea as a 
whole, the team agreed to adopt this intervention with 
some modifications. As a next step, the Fresno team is 
building upon their early success by continuing to use 
Improvement Science and their Cycle of Continuous 
Improvement Learning and Competency Tool to 
address barriers to college matriculation. Next, they 
are hoping to develop and test interventions that will 
counteract the "summer melt" phenomenon that 
often results in lower college attendance by students 
who are admitted and intending to enroll at a 
college/university. 

Lessons Learned: Implementing 
Continuous Improvement at 
Scale 

The team in Fresno had some early wins in using the 
principles of Improvement Science to boost college 
access, but they have far more work to do to make 
dramatic changes within the district. The team 
acknowledges that true continuous improvement 
requires a culture change within districts, as it requires 
a level of discipline and rigor that is not yet the norm 
in education environments. The Equity and Access 
team has learned a great deal about building staff 
capacity to engage in continuous improvement 
initiatives. 

Define What Your Team or District Means by 
"Continuous Improvement" 

The Fresno team learned that there was not a 
common language or definition or continuous 
improvement across their team. This lack of 
consensus caused staff to feel as though they were 
doing continuous improvement even when they were 
not adhering to the principles that the team was trying 
to adopt. This created a number of problems for the 
team, the largest of which was that staff often wanted 
to adopt new terminology without actually changing 
their existing practices. The Equity and Access team 
used the principles of Improvement Science to apply a 
disciplined and rigorous methodology to their work. 
They developed the Cycle of Continuous 
Improvement Learning and Competency Tool as a 
way to develop a shared language and structure 
across their own team and future district teams. The 
tool holds the team accountable to their own change 
ideas and facilitates the implementation of principles 
of continuous improvement that they have agreed will 
help determine whether they have improved their 
system over time. 





Anchor All Work Around the Problem of Practice 

Doing so provides structure and discipline. It tempers 
the desire to implement solutions without 
demonstrating a solid base of understanding of: (a) the 
problem being solved; (b) academic, technical, and/or 
clinical expertise that informs the problem being solved; 
or (c) a working theory of improvement for the problem 
being solved. The Equity and Access team made 
concentrated efforts to better understand why eligible 
students weren't applying to certain colleges and 
universities. They involved school counselors to make 
sure that the root causes they were identifying could 
actually explain the student behaviors reflected in their 
data. Once the team had a clear understanding of the 
problem and what an improved outcome would look 
like, they selected a change idea (college packets) that 
they believed would address one of the primary root 
causes of the problem of practice. Throughout their 
PDSA cycles, the team kept a laser focus on this 
problem of practice. 

Test Before You Scale and Use Data to Determine 
if Tests Are Actual Improvements 

Documentation can be a useful tool to ensure clarity, 
accountability, and transparency. In Fresno, their Cycle 
of Continuous Improvement Learning and 
Competency Tool required that staff provide research 
and evidence to (a) support their change idea, (b) 
clarify which individuals had taken ownership over 
specific tasks, and (c) mandate that staff take stock of 
their success and challenges. Further, such 
documentation enhanced transparency of these efforts 
to district leaders, the research community, and even 
the public. 

Balance Science and Practice 

An important consideration in this work is the need to 
balance fidelity to the "science" of Improvement 
Science with the urgency for change. Close adherence 
to the six Improvement Science principles can be 
difficult when promising change opportunities present 
themselves with short action timelines. The FUSD team 
posed two guiding questions as they sought to strike 
this balance. Had peer reviewers verified that the 
solution under consideration was viable and rigorous? 
Was the opportunity to "change conditions in the 
present" strong enough to outweigh the impact of 
deviating from the agreed upon practices and 
processes of the team? 



Don't Be Afraid to Fail 

The theory of change behind continuous improvement 
is that school systems must become learning 
organizations that constantly test interventions 
designed to address systemic problems and then learn 
to determine whether these changes bring about 



improvements. Since districts are testing change 
hypotheses, there is a strong likelihood that some of 
these hypotheses will not produce the intended 
outcomes. Fresno's Equity and Access leadership team 
continues to wrestle with daily failures, even though 
they are a natural part of continuous improvement. If 
the process of improvement is structured correctly, 
failure produces as much learning as success. 

Stay Humble 

In trying to build the will and skill of colleagues to 
engage in Improvement Science work, the FUSD team 
recognized a common attitude: that they already had 
the expertise to solve their performance problem. 
Upon reflection, the team accepted that they lacked 
the humility to be honest about why, despite in-house 
expertise, their outcomes continued to be 
unsatisfactory. To combat this mindset, the Equity and 
Access team constantly asked themselves and others 
in the district, "If we know how to solve this problem, 
then why haven't we yet?" 

Fresno Unified School District began its Improvement 
Science journey by starting small and building the 
capacity of one team within the district to improve a 
narrow problem of practice for a subgroup of its 
students. This early work has resulted in improved 
outcomes for a subgroup of Fresno's students as well 
as the development of a learning tool that will 
ultimately help the district build Improvement Science 
capacity across more of its teams and departments. 



' California's School Dashboard can be accessed at 
www.caschooidashboard.org 
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